Method and System for Determining Information on an Expected Trajectory of an Object

ABSTRACT

A method for determining information on an expected trajectory of an object comprises: determining input data being related to the expected trajectory of the object; determining first intermediate data based on the input data using a machine-learning method; determining second intermediate data based on the input data using a model-based method; and determining the information on the expected trajectory of the object based on the first intermediate data and based on the second intermediate data.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to European Patent Application Number20159899.2, filed Feb. 27, 2020, the disclosure of which is herebyincorporated by reference in its entirety herein.

BACKGROUND

The present disclosure relates to methods and systems for determininginformation on an expected trajectory of an object.

A human driver of a vehicle considers the surrounding trafficparticipants to make maneuver decision. The human driver anticipates thefuture trajectories of the surrounding dynamic objects and the potentialrisk of collision subconsciously, constantly and instantly. At the sametime, the human driver tries to follow the lane, and keep the vehicle inthe center of the lane.

For at least partially autonomous vehicles, the vehicle itself isproposed to carry out all this processing, which requires information onthe surrounding of the vehicle and of the vehicle itself, and inparticular requires information on expected trajectories of variousobjects.

Accordingly, there is a need to efficiently and reliable determineinformation on an expected trajectory of an object.

SUMMARY

The present disclosure provides a computer-implemented method, acomputer system and a non-transitory computer readable medium accordingto the independent claims. Embodiments are given in the subclaims, thedescription and the drawings.

In one aspect, the present disclosure is directed at acomputer-implemented method for determining information on an expectedtrajectory of an object (for example a vehicle or a pedestrian), themethod comprising the following steps performed (in other words: carriedout) by computer hardware components: determining input data beingrelated to the expected trajectory of the object; determining firstintermediate data based on the input data using a machine-learningmethod; determining second intermediate data based on the input datausing a model-based method; and determining the information on theexpected trajectory of the object based on the first intermediate dataand based on the second intermediate data.

The expected trajectory may be a future expected trajectory, wherein“future” may be understood as to be after the input data. This may beunderstood as predicting a trajectory in the future.

In another aspect, the expected trajectory may be a past trajectory,which may be understood to be before the input data. This may beunderstood as reconstructing a trajectory in the past.

In other words, information on an expected trajectory may be determinedbased on the results of two different methods, one method being amodel-based method, and the other method being a machine-learningmethod. The method provides a fused (fusing the results of themodel-based method and the results of the machine-learning method)trajectory prediction method. A corresponding system may also beprovided.

Determining the information on the expected trajectory of the objectbased on the first intermediate data and based on the secondintermediate data may be understood as fusing the first intermediatedata and the second intermediate data to obtain (directly or after oneor more further processing steps) the information on the expectedtrajectory of the object.

Illustratively, the machine-learning method is not required to learneverything, but only the part that the model-based method (for exampleof an existing prediction system) does not describe. For example, theinteraction with other vehicles cannot easily be modelled, and thus, theaspect of interaction with other vehicles may be learned by themachine-learning method.

For example, if an existing systems uses “velocity×time” to predict thefuture position, it can obviously not describe the acceleration andbreaking of a vehicle movement accurately. Then the machine-learningmethod may be trained to take into account the acceleration and breakingof the vehicle movement.

With the method according to this aspect, the machine learning approachmay be incorporated with known knowledge (i.e. the model-based method)and may improve the performance for long-term trajectory application.

According to another aspect, the input data may be based on sensor data,wherein the sensor data comprising at least one of radar data, lidardata, ultrasound data, mobile radio communication data or camera data.Using those sensors, the previous trajectory of the object may bedetermined, and the information of the previous trajectory may be usedas the input data.

According to another aspect, the input data comprises informationrelated to positions and/or velocities and/or accelerations of theobject and/or further objects.

According to another aspect, the information on the expected trajectoryof the object is represented by discrete points of time (wherein theactual expected trajectory may be determined based on interpolationbetween the discrete points) or by a continuous function over time(wherein the actual expected trajectory may be determined based onevaluating the continuous function).

According to another aspect, the information on the expected trajectoryof the object is determined based on adding the first intermediate dataand the second intermediate data. The first intermediate data and thesecond intermediate data may be weighted, and the weight may be providedby a trained fusion network. The weight may be fixed, or may bedependent on the first intermediate data and/or the second intermediatedata.

It will be understood that any network referred to herein may be aneural network or any other suitable kind of network for determiningoutput data based on input data.

According to another aspect, the information on the expected trajectoryof the object may be determined using a fusion network based on thefirst intermediate data and based on the second intermediate data. Thefusion network may be a neural network, and may be trained to combinethe first intermediate data and the second intermediate data to providethe expected trajectory.

According to another aspect, the fusion network comprises a plurality oflearnable parameters. For example, the fusion network may be a neuralnetwork, and the learnable parameters may be weights of a neuralnetwork.

According to another aspect, the fusion network comprises a weightedsum. The weights used in the weighted sum may be constant or may belearned from the first intermediate data and/or the second intermediatedata (for example using a neural network).

According to another aspect, the information on the expected trajectoryof the object is determined based on adding the first intermediate dataand the second intermediate data.

According to another aspect, the machine-learning method comprises afirst encoding network. According to another aspect, the model-basedmethod comprises a second encoding network.

According to another aspect, first decoded intermediate data may bedetermined based on the first intermediate data using a first decodingnetwork, wherein the information on the expected trajectory of theobject is determined further based on the first decoded intermediatedata. According to another aspect, second decoded intermediate data maybe determined based on the second intermediate data using a seconddecoding network, wherein the information on the expected trajectory ofthe object is determined further based on the second decodedintermediate data.

According to another aspect, the first intermediate data is determinedusing a first encoding network, the second intermediate data isdetermined using a second encoding network, and the information on theexpected trajectory is determined using a fusion network based on thefirst intermediate data and the second intermediate data. In otherwords, the fusion may be carried out in the latent space (as will bedescribed in more detail below), and the first and second intermediatedata are the encoding outputs.

According to another aspect, the first intermediate data is determinedusing a first encoding network and a first decoding network, the secondintermediate data is determined using a second encoding network and asecond decoding network, and the information on the expected trajectoryis determined using a fusion network based on the first intermediatedata and the second intermediate data. In other words, the fusion may becarried out in the physical space (as will be described in more detailbelow), and the first and second intermediate data are the decodingoutputs.

In another aspect, the present disclosure is directed at a computersystem, said computer system comprising a plurality of computer hardwarecomponents configured to carry out several or all steps of thecomputer-implemented method described herein.

The computer system may comprise a plurality of computer hardwarecomponents (for example a processing unit, at least one memory unit andat least one non-transitory data storage). It will be understood thatfurther computer hardware components may be provided and used forcarrying out steps of the computer-implemented method in the computersystem. The non-transitory data storage and/or the memory unit maycomprise a computer program for instructing the computer to performseveral or all steps or aspects of the computer-implemented methoddescribed herein, for example using the processing unit and the at leastone memory unit.

In another aspect, the present disclosure is directed at a vehiclecomprising the computer system as described above.

In another aspect, the vehicle further comprises at least one sensorconfigured to acquire sensor data, wherein the computer system isconfigured to determine the input data based on the acquired sensordata.

In another aspect, the present disclosure is directed at anon-transitory computer readable medium comprising instructions forcarrying out several or all steps or aspects of the computer-implementedmethod described herein. The computer readable medium may be configuredas: an optical medium, such as a compact disc (CD) or a digitalversatile disk (DVD); a magnetic medium, such as a hard disk drive(HDD); a solid state drive (SSD); a read only memory (ROM), such as aflash memory; or the like. Furthermore, the computer readable medium maybe configured as a data storage that is accessible via a dataconnection, such as an internet connection. The computer readable mediummay, for example, be an online data repository or a cloud storage.

The present disclosure is also directed at a computer program forinstructing a computer to perform several or all steps or aspects of thecomputer-implemented method described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments and functions of the present disclosure aredescribed herein in conjunction with the following drawings, showingschematically:

FIG. 1 an illustration of a machine learning based approach;

FIG. 2 an illustration of an extended system according to variousembodiments;

FIG. 3 an illustration of a system according to various embodiments withan alternative arrangement to the arrangement shown in FIG. 2;

FIG. 4 an illustration of a system according to various embodiments withan alternative arrangement to the arrangement shown in FIG. 2; and

FIG. 5 a flow diagram illustrating a method for determining informationon an expected trajectory of an object according to various embodiments.

DETAILED DESCRIPTION

FIG. 1 shows an illustration 100 of a machine learning based approach.An input X 102, which may include measurement features of the targettrajectory, and possibly also its neighboring vehicles' trajectories,may be provided to an encode network 104. The measurements may forexample be information on current positions and velocities. The encodenetwork 104 may be a neural network (NN), for example a RNN (recurrentneural network)-based network. For example, the approach described inEuropean Patent Application 19202631.8, which is incorporated herein byreference in its entirety, may be used as the encode network 104. Theoutput of this network 106 (which may also be referred to as h1) may bea code in the latent space (in other words: hidden space), describingthe complete past dynamic of the target, and possibly also itsinteraction with surrounding vehicles.

With the latent code 106, the decode network 108 may decode it into thegiven form of the predicted future trajectory Y 110. In an example, fivepoints may be predicted for a five second long future trajectory, withone second interval among them.

A past trajectory of an object may be described with multiple points atconsecutive time frames. Each point may consist of, but is notrestricted to, features (such as for example position, velocities,acceleration, heading angles) to describe the state of the object atspecific past time frames.

According to various embodiments, given the past trajectory of thetarget object, and possibly also its surrounding objects' trajectories,the future trajectory of the target may be predicted, which may besimilarly represented by multiple points at multiple future time frames,or which may be described in a continuous way, such as the methoddescribed in European Patent Application 19219051.0, which isincorporated herein by reference in its entirety.

FIG. 2 shows an illustration 200 of an extended system according tovarious embodiments, which extends the pure machine-learning basednetwork of FIG. 1.

X 202 may be the input, for example system trajectory input (in otherwords: the past trajectory). Y 222 may be the predicted futuretrajectory. The first encoding network 204 and the first latent code 206(h1) may be similar or identical to the encode network 104 and latentcode 106 of FIG. 1. The first latent code 206 may be first intermediatedata.

The model-based prediction system 208 may output model-based results210, which may be used as input to a second encoding network 212. Thesecond encoding network 212 may output a second latent code 214. Thesecond latent code 214 may be second intermediate data. The secondencoding network 212 may be used to encode the output 210 of themodel-based prediction (which, for example, may include or may be thefuture trajectory points) into a latent code, so that the fusion may becarried out in the latent space.

The fusion network 216 may be a network with learnable (in other words:trainable or optimizable) parameters, and may output a fused code 218(h) based on the first latent code 206 (h1) and the second latent code214 (h2). For example, the fusion network 216 may determine the fusedcode 218 according to the following equation:

h=f(W1*h1+W2*h2+b),

wherein f may be a function, for example an activation function, and W1and W2 may be trainable weights. The fused code 218 may be provided to adecoder network 220 to decode the final trajectory.

The trajectory data may be a time series data, and the encoding anddecoding networks may be implemented using RNNs, for example longshort-term memory (LSTM) and/or gated recurrent unit (GRU). However, itwill be understood that instead of an RNN, any other type of network maybe used for encoding or decoding, for example a convolutional neuralnetwork (CNN) may be used.

The input data X 202 may, for example, include the past trajectory pointposition(s), the measured velocity or velocities and/or acceleration(s)in longitudinal and/or lateral directions. These data may be provided inthe vehicle coordinate system. Furthermore, the input data X may includethe ego vehicle's own velocity and/or yaw rate.

The latent code, as its name suggests, may not have a specific physicalmeaning. The latent code may be the output of an encoding network, andthis output does not really have physical meaning, so that this outputmay be considered as a latent code. The latent code may be consideredsimilar to an encrypted code.

The model-based results 210 may include data representing the positionsof the predicted trajectory points, for example x and y positions. Invarious embodiments, the predicted trajectory points may have morevariables than just positions, for example, it can also include thevelocities and/or accelerations.

The model-based prediction system 208 may include a computational,physics-based model for prediction of movement or a trajectory, and maybe based on physical laws or may be tracking-based.

For example, there are physical laws restricting and describing themovement of an object. Theoretically one can use the physical laws topredict the movement of any moving objects.

As a simplified example, given speed v, acceleration a, the drivendistance s after t seconds can be calculated using:

s=v*t+0.5*a*t{circumflex over ( )}2

This physical law is correct, but taking only this physical law may notwork properly for prediction of moving objects due to the underlyingassumption of the velocity v and acceleration a being kept constant(i.e. the same) for the time period of t (for example t seconds), andthis assumption may not be valid or feasible for a relative long periodof t, which is the case for long-term trajectory prediction. If it isdesired to predict the position for the next e.g. 0.05 second of avehicle, the above physical law may hold well. A vehicle cannot changeits movement just instantly, so for a 0.05 second future prediction, theassumption can still be feasible. But for the autonomous drivingapplications, a long-term prediction for a few seconds is needed, forexample to help anticipates the possible danger, and/or to help planningthe ego path.

A human driver may react to a sudden emergency quite fast and hit thebreak in less than half second, so such a simple physical law basedapproach obviously may not be used for autonomous driving applications.

As such, a more advanced physical law based approach may be used for themodel-based method according to various embodiments. For example, themovement may be decomposed into longitudinal and lateral direction andthe x and y components may be predicted separately, using velocity andacceleration. If more information are provided, so not only velocity andacceleration but also e.g. jerk, turning rate, pitch angle, etc. themodel-based prediction may be further refined. The real world drivingcondition may deviate from what is described by the above equation.

In the object tracking area, the Kalman filter based approaches forstate estimation may be used. For example, such approaches may be usedfor radar applications.

A tracking system may incorporate a system model, which describes themovement of an object, e.g. with complicated physical modeling, withvarious parameters. The tracker may estimate these parameters given themeasurement it gets.

Furthermore, techniques like Multiple Model (MM), for exampleInteracting Multiple Model (IMM), may provide the possibility ofincorporating multiple different system models into a tracking system.Such trackers may estimate the possibilities of all the models togetherwith their corresponding parameters.

As a simplified example, a vehicle tracking system may have thefollowing models: the vehicle is driving straight forward with constantacceleration (constant velocity is a special case that acceleration isconstantly 0). The vehicle is turning with constant yaw movement. Thenthe tracking system may estimate the possibilities of each of the modelsand see which one is most likely happening at current time frame. SuchIMM based tracking system may respond to a maneuver more quickly and maythus provide good performance in real world applications.

Such a system may be used also for prediction, since it has multiplemodels and their possibility, it can give better prediction.

According to European Patent Application 19202631.8, which isincorporated herein by reference in its entirety, three key factors maybe categorized for long-term trajectory prediction: past trajectory,dynamic context, and static context.

The factor of “past trajectory”, namely how to model and estimate themovement more accurately, may be considered. According to variousembodiments, for a long-term future trajectory, also the interactionwith the surrounding dynamic and static context may be considered.

The physical laws to describe the movement may not be wrong, just theassumption they use may be not be valid for all situations. According tovarious embodiments, the knowledge of the models involving physical lawsmay be used, and a machine-learning method may be trained to compensatefor the invalidity of the physical model at certain situations.

According to various embodiments, the machine-learning based approachmay be combined with any exist prediction system: at one hand, the datadriven approach may be incorporated into the problem, avoid explicitmodeling. The movement of an object may still be modeled per physicallaws, but other factor of the context may not (easily) be done in thatway. Thus, a machine-learning approach according to various embodimentsto approximate the underlying rather complex interaction with contextmay be used.

On the other hand, the known knowledge from any existing system may bebrought into the machine-learning approach.

According to various embodiments, integration of any existing system(for example model-based method) with a machine learning based approachfor trajectory prediction may be provided. Regardless of how simple orcomplicated the existing system is, the machine learning base approach(for example the (neural) network) may help to improve the trajectoryprediction results.

Returning to FIG. 2, the processing path “X->encoder network1->h1->fusion network” may integrate any machine learning system. Forexample, a system that learns the interaction among the target and itssurrounding vehicles (for example as described in European PatentApplication 19202631.8, which is incorporated herein by reference in itsentirety), or a system learns the interaction of the target with thestatic driving environment.

The process path “X2->encode network 2->h2->fusion” may bring knownknowledge into the machine learning approach, for example using themodel-based prediction system 208. The overall network does not need tolearn everything from data completely.

FIG. 3 shows an illustration 300 of a system according to variousembodiments with an alternative arrangement to the arrangement shown inFIG. 2. For example, a first decoding network 302 may provide a firstportion 304 (y1) of the predicted future trajectory 222, based on thefirst latent code 206, and a second decoding network 306 may provide asecond portion 308 (y2) of the predicted future trajectory 222, based onthe second latent code 214. The predicted future trajectory 222 may be aweighted sum 310 of the first portion 304 and the second portion (whichcorresponds to the model-based results 210), so that: y=w*y1+(1−w)*y2,wherein the weight w 314 of the weighted addition may be learned by thefusion network 312.

When comparing FIG. 2 and FIG. 3, it can be seen that the first latentcode 206 and the second latent code 214 are generated the same way. Adifference lies in how the first and second latent codes 206, 214 arecombined and finally decoded into the final result Y (222).

In the embodiment illustrated in FIG. 2, fusion is carried out in thelatent space, i.e. the fusion network 216 is directly combining thefirst latent codes 206 and the second latent code 214 into one latentcode h (218). Decoding of the latent code h (218) is carried outafterwards, using the decoding network 220. In the embodimentillustrated in FIG. 2, the first intermediate data may be the firstlatent code 206, and the second intermediate data may be the secondlatent code 214.

In the embodiments illustrated in FIG. 3, fusion is carried out in thephysical space, i.e. the predicted trajectories y1 (304) and y2 (308)are combined per weighted sum (instead of their latent codes as in FIG.2). The weight w (218) for adding may be learned from the first andsecond latent codes 206, 214. In the embodiment illustrated in FIG. 2,the first intermediate data may be the first portion of the predictedfuture trajectory y1 (304), and the second intermediate data may be thesecond portion of the predicted future trajectory y2 (304).

According to various embodiments, the second decoding network 306 maynot be present, and the model-based results 210 may be used instead ofthe predicted trajectory y2 (308).

FIG. 4 shows an illustration 400 of a system according to variousembodiments with an alternative arrangement to the arrangement shown inFIG. 2, wherein the fusion may be an “add” (addition) operation. Giventhe input, the network gives the prediction 304 (y1), and this isdirectly added to the model-based prediction 301. This may intuitivelybe understood as the network making corrections to the model-basedpredictions.

The embodiment illustrated in FIG. 4 may be considered as a special caseof the embodiment illustrated in FIG. 3, where the weight w has aconstant value of “1”. Thus, the second encoding network 212, the seconddecoding network 306, and the fusion network 312 for generating theweight w 314 may not be needed any more.

Thus, the main difference between the embodiments illustrated in FIG. 2and FIG. 3 or FIG. 4 is where the fusion is carried out (either in thelatent space by fusing the latent codes, or in the physical space bydecoding the codes first and adding them together afterwards).

FIG. 5 shows a flow diagram 500 illustrating a method for determininginformation on an expected trajectory of an object according to variousembodiments. In 502, input data being related to the expected trajectoryof the object may be determined. In 504, first intermediate data may bedetermined based on the input data using a machine-learning method. In506, second intermediate data may be determined based on the input datausing a model-based method. In 508, the information on the expectedtrajectory of the object may be determined based on the firstintermediate data and based on the second intermediate data.

According to various embodiments, the input data may be based on sensordata, the sensor data comprising at least one of radar data, lidar data,ultrasound data, mobile radio communication data or camera data.

According to various embodiments, the input data may include or may beinformation related to positions and/or velocities and/or accelerationsof the object and/or further objects.

According to various embodiments, the information on the expectedtrajectory of the object may be represented by discrete points of timeor by a continuous function over time.

According to various embodiments, the information on the expectedtrajectory of the object may be determined based on adding the firstintermediate data and the second intermediate data.

According to various embodiments, the information on the expectedtrajectory of the object may be determined using a fusion network basedon the first intermediate data and based on the second intermediatedata.

According to various embodiments, wherein the fusion network may includea plurality of learnable parameters.

According to various embodiments, the fusion network comprises aweighted sum (in other words: a weighted sum is used in (or as) thefusion network).

According to various embodiments, the information on the expectedtrajectory of the object is determined based on adding the firstintermediate data and the second intermediate data.

According to various embodiments, the machine-learning method mayinclude or may be a first encoding network.

According to various embodiments, the model-based method may include asecond encoding network.

According to various embodiments, the first intermediate data may bedetermined using a first encoding network, the second intermediate datamay be determined using a second encoding network, and the informationon the expected trajectory may be determined using a fusion networkbased on the first intermediate data and the second intermediate data.

According to various embodiments, the first intermediate data may bedetermined using a first encoding network and a first decoding network,the second intermediate data may be determined using a second encodingnetwork and a second decoding network, and the information on theexpected trajectory may be determined using a fusion network based onthe first intermediate data and the second intermediate data.

According to various embodiments, the method may further includedetermining first decoded intermediate data based on the firstintermediate data using a first decoding network, wherein theinformation on the expected trajectory of the object may be determinedfurther based on the first decoded intermediate data.

According to various embodiments, the method may further includedetermining second decoded intermediate data based on the secondintermediate data using a second decoding network, wherein theinformation on the expected trajectory of the object may be determinedfurther based on the second decoded intermediate data.

Each of the steps 502, 504, 506, 508 and the further steps describedabove may be performed by computer hardware components.

What is claimed is:
 1. A computer-implemented method, the methodcomprising: determining, by computer hardware components, information onan expected trajectory of an object by at least: determining input databeing related to the expected trajectory of the object; determiningfirst intermediate data based on the input data using a machine-learningmethod; determining second intermediate data based on the input datausing a model-based method; and determining the information on theexpected trajectory of the object based on the first intermediate dataand based on the second intermediate data.
 2. The computer-implementedmethod of claim 1, wherein the information on the expected trajectory ofthe object is determined using a fusion network based on the firstintermediate data and based on the second intermediate data.
 3. Thecomputer-implemented method of claim 2, wherein the fusion networkcomprises a plurality of learnable parameters.
 4. Thecomputer-implemented method of claim 2, wherein the fusion networkcomprises a weighted sum.
 5. The computer-implemented method of claim 1,wherein the information on the expected trajectory of the object isdetermined based on adding the first intermediate data and the secondintermediate data.
 6. The computer-implemented method of claim 1,wherein the machine-learning method comprises a first encoding network.7. The computer-implemented method of claim 1, wherein the model-basedmethod comprises a second encoding network.
 8. The computer-implementedmethod of claim 1, wherein the first intermediate data is determinedusing a first encoding network; wherein the second intermediate data isdetermined using a second encoding network; wherein the information onthe expected trajectory is determined using a fusion network based onthe first intermediate data and the second intermediate data.
 9. Thecomputer-implemented method of claim 1, wherein the first intermediatedata is determined using a first encoding network and a first decodingnetwork; wherein the second intermediate data is determined using asecond encoding network and a second decoding network; wherein theinformation on the expected trajectory is determined using a fusionnetwork based on the first intermediate data and the second intermediatedata.
 10. The computer-implemented method of claim 1, wherein the inputdata is based on sensor data, the sensor data comprising at least one ofradar data, lidar data, ultrasound data, mobile radio communication dataor camera data.
 11. The computer-implemented method of claim 1, whereinthe input data comprises information related to positions and/orvelocities and/or accelerations of the object and/or further objects.12. A system, the system comprising: a computer system comprising aplurality of computer hardware components configured to: determineinformation on an expected trajectory of an object by at least:determining input data being related to the expected trajectory of theobject; determining first intermediate data based on the input datausing a machine-learning method; determining second intermediate databased on the input data using a model-based method; and determining theinformation on the expected trajectory of the object based on the firstintermediate data and based on the second intermediate data.
 13. Thesystem of claim 12, further comprising: a vehicle comprising thecomputer system.
 14. The system of claim 13, wherein the vehicle furthercomprises at least one sensor configured to acquire sensor data; andwherein the computer system is configured to determine the input databased on the acquired sensor data.
 15. The system of claim 12, whereinthe information on the expected trajectory of the object is determinedusing a fusion network based on the first intermediate data and based onthe second intermediate data.
 16. The system of claim 15, wherein thefusion network comprises a plurality of learnable parameters.
 17. Thesystem of claim 15, wherein the fusion network comprises a weighted sum.18. The system of claim 13, wherein the information on the expectedtrajectory of the object is determined based on adding the firstintermediate data and the second intermediate data.
 19. The system ofclaim 13, wherein the machine-learning method comprises a first encodingnetwork; and wherein the model-based method comprises a second encodingnetwork.
 20. A non-transitory computer readable medium comprisinginstructions, that when executed, configure computer hardware componentsto: determine information on an expected trajectory of an object by atleast: determining input data being related to the expected trajectoryof the object; determining first intermediate data based on the inputdata using a machine-learning method; determining second intermediatedata based on the input data using a model-based method; and determiningthe information on the expected trajectory of the object based on thefirst intermediate data and based on the second intermediate data.